An order optimal policy for exploiting idle spectrum in cognitive radio networks

摘要

In this paper a spectrum sensing policy employing recency-based explorationis proposed for cognitive radio networks. We formulate the problem of finding aspectrum sensing policy for multi-band dynamic spectrum access as a stochasticrestless multi-armed bandit problem with stationary unknown rewarddistributions. In cognitive radio networks the multi-armed bandit problemarises when deciding where in the radio spectrum to look for idle frequenciesthat could be efficiently exploited for data transmission. We consider twomodels for the dynamics of the frequency bands: 1) the independent model wherethe state of the band evolves randomly independently from the past and 2) theGilbert-Elliot model, where the states evolve according to a 2-state Markovchain. It is shown that in these conditions the proposed sensing policy attainsasymptotically logarithmic weak regret. The policy proposed in this paper is anindex policy, in which the index of a frequency band is comprised of a samplemean term and a recency-based exploration bonus term. The sample mean promotesspectrum exploitation whereas the exploration bonus encourages for furtherexploration for idle bands providing high data rates. The proposed recencybased approach readily allows constructing the exploration bonus such that itwill grow the time interval between consecutive sensing time instants of asuboptimal band exponentially, which then leads to logarithmically increasingweak regret. Simulation results confirming logarithmic weak regret arepresented and it is found that the proposed policy provides often improvedperformance at low complexity over other state-of-the-art policies in theliterature.